OSAC-853: add AAP presubmit e2e-vmaas job by omer-vishlitzky · Pull Request #79365 · openshift/release

omer-vishlitzky · 2026-05-16T12:08:50Z

Summary

Add e2e-vmaas presubmit test for osac-aap PRs, matching the pattern used by fulfillment-service and osac-operator
Build the execution environment (EE) image from the PR source in CI
Override AAP_PROJECT_GIT_BRANCH and AAP_EE_IMAGE in the kustomize overlay so AAP uses the PR's playbooks and EE instead of the pinned versions

Why AAP is different

Unlike other OSAC components (single container image), AAP has three independently-versioned artifacts that must all come from the PR:

Artifact	How it's overridden
EE image (runtime + collections)	`kustomize edit set image` (existing mechanism) + `AAP_EE_IMAGE` sed
Playbooks (git-synced by AAP)	`AAP_PROJECT_GIT_BRANCH` sed to PR head SHA
Config manifests (K8s resources)	`COPY config/` into installer (existing mechanism)

The boot script detects a .aap-source-sha marker file in the installer image. When present, it overrides the git ref and EE image. When absent (all other repos), it's a no-op.

Jira: https://redhat.atlassian.net/browse/OSAC-853

Test plan

Rehearsal jobs pass on this PR
Open a test PR on osac-aap and verify the e2e-vmaas job triggers and the AAP overrides are applied

This PR adds an e2e-vmaas presubmit for osac-aap in OpenShift CI that builds and runs tests against the PR's artifacts (execution environment image, playbooks git ref, and installer config) rather than pinned/released artifacts.

What changed (practical impact)

Repos/config affected: ci-operator configuration for osac-project/osac-aap and the osac-project cluster-tool boot step in the step-registry. This changes how osac-aap PRs are built and how the cluster-tool boots and refreshes AAP for e2e-vmaas runs.
Effect: CI will build PR-specific EE and installer images, embed a source-SHA marker in the installer image, and the cluster-tool boot/refresh will apply kustomize/sed overrides so the cluster uses the PR’s playbooks (git ref), EE image, and config manifests during the e2e-vmaas job.

Key changes

ci-operator config (ci-operator/config/osac-project/osac-aap/osac-project-osac-aap-main.yaml)
- Adds base_images (dev-scripts, origin-cli, osac-installer, osac-test-infra) and binary_build_commands that write a source-SHA file from the PR build.
- Adds two PR-specific image builds:
  - osac-aap-pr: builds the PR execution environment (CentOS Stream 9 base, Python 3.12, runtime/dev deps, pip requirements, Ansible collections/vendor) and copies oc/kubectl from origin-cli.
  - osac-installer-with-pr: builds an installer image based on osac-installer, copies config/ into /installer/base/osac-aap/config/, and writes the generated source-sha into /installer/.aap-source-sha.
- Ensures PR-specific images are excluded from promotion to releases.latest and introduces a releases.latest.candidate block for nightly ocp 4.20 testing.
- Adds an e2e-vmaas presubmit wired to osac-aap-pr, osac-installer-with-pr, and osac-test-infra; sets COMPONENT_IMAGE_NAME and uses the osac-project-cluster-tool-vmaas workflow.
Cluster-tool boot script (ci-operator/step-registry/osac-project/cluster-tool/boot/osac-project-cluster-tool-boot-commands.sh)
- Detects /installer/.aap-source-sha in the installer image at runtime.
- When present, constructs sed/kustomize overrides to set AAP_PROJECT_GIT_BRANCH to the detected SHA so AAP will git-sync playbooks from the PR head.
- When a component image override is supplied, also sets AAP_EE_IMAGE to the PR-built EE image.
- Applies these overrides during the refresh phase so the running cluster uses the PR’s playbooks, EE image, and installer-provided config manifests.

Testing / behavior

Rehearsal jobs pass on this PR.
Expected behavior: the CI builds the PR-specific EE and installer images; the installer’s .aap-source-sha marker triggers the boot script to override AAP_PROJECT_GIT_BRANCH and (when applicable) AAP_EE_IMAGE, enabling e2e-vmaas tests to run against the PR’s playbooks, EE, and config.

Notes

Follows the pattern used by other OSAC components (fulfillment-service, osac-operator) to coordinate three independently-versioned artifacts: EE image, playbooks (git ref), and config manifests.
PR-specific images are intentionally not promoted into releases.

openshift-ci-robot · 2026-05-16T12:08:54Z

@omer-vishlitzky: This pull request references OSAC-853 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the task to target the "5.0.0" version, but no target version was set.

Details

In response to this:

Summary

Add e2e-vmaas presubmit test for osac-aap PRs, matching the pattern used by fulfillment-service and osac-operator

Build the execution environment (EE) image from the PR source in CI

Override AAP_PROJECT_GIT_BRANCH and AAP_EE_IMAGE in the kustomize overlay so AAP uses the PR's playbooks and EE instead of the pinned versions

Why AAP is different

Unlike other OSAC components (single container image), AAP has three independently-versioned artifacts that must all come from the PR:

Artifact How it's overridden

EE image (runtime + collections) kustomize edit set image (existing mechanism) + AAP_EE_IMAGE sed

Playbooks (git-synced by AAP) AAP_PROJECT_GIT_BRANCH sed to PR head SHA

Config manifests (K8s resources) COPY config/ into installer (existing mechanism)

The boot script detects a .aap-source-sha marker file in the installer image. When present, it overrides the git ref and EE image. When absent (all other repos), it's a no-op.

Jira: https://redhat.atlassian.net/browse/OSAC-853

Test plan

Rehearsal jobs pass on this PR

Open a test PR on osac-aap and verify the e2e-vmaas job triggers and the AAP overrides are applied

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

coderabbitai · 2026-05-16T12:09:15Z

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

@coderabbitai resume to resume automatic reviews.
@coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

▶️ Resume reviews
🔍 Trigger review

Walkthrough

This PR embeds a build source SHA into two PR images, updates promotion/release/test wiring, and updates the cluster boot script to read that SHA and apply sed-based AAP overrides during the refresh step.

Changes

AAP Source SHA Tracking and Override Mechanism

Layer / File(s)	Summary
Base images and source SHA generation `ci-operator/config/osac-project/osac-aap/osac-project-osac-aap-main.yaml`	Base image references for dev-scripts, origin-cli, osac-installer, and osac-test-infra are defined; `binary_build_commands` generates `/tmp/source-sha` from git revision with fallback.
PR image builds with source SHA embedding `ci-operator/config/osac-project/osac-aap/osac-project-osac-aap-main.yaml`	`osac-aap-pr` dockerfile builds CentOS Stream 9 with Python 3.12, build/runtime dependencies, Ansible collections/vendor, and oc/kubectl; `osac-installer-with-pr` copies installer config and the generated source-sha file into the installer image.
Promotion rules and release candidate configuration `ci-operator/config/osac-project/osac-aap/osac-project-osac-aap-main.yaml`	Latest namespace promotion is updated to exclude the new PR images; release candidate targeting OCP nightly stream version 4.20 is added.
E2E test wiring `ci-operator/config/osac-project/osac-aap/osac-project-osac-aap-main.yaml`	`e2e-vmaas` test entry declares dependencies on `osac-aap-pr`, `osac-installer-with-pr`, and `osac-test-infra`, specifies the workflow `osac-project-cluster-tool-vmaas`, and defines `COMPONENT_IMAGE_NAME`.
Boot script source SHA detection and AAP overrides `ci-operator/step-registry/osac-project/cluster-tool/boot/osac-project-cluster-tool-boot-commands.sh`	Boot script reads `.aap-source-sha` from the installer image and constructs `sed` commands to override `AAP_PROJECT_GIT_BRANCH`; optionally adds `AAP_EE_IMAGE` override when `COMPONENT_IMAGE` is set.
Refresh command with combined overrides `ci-operator/step-registry/osac-project/cluster-tool/boot/osac-project-cluster-tool-boot-commands.sh`	Cluster refresh invocation is updated to apply both `COMPONENT_OVERRIDE_CMD` and the newly added `AAP_OVERRIDE_CMD` before executing the refresh script.

Sequence Diagram(s)

sequenceDiagram
  participant CI as CI Builder
  participant Installer as installer image (osac-installer-with-pr)
  participant Boot as cluster boot script
  participant Refresh as scripts/refresh-after-snapshot.sh
  CI->>Installer: build image and embed .aap-source-sha
  Boot->>Installer: read .aap-source-sha from image
  Boot->>Boot: construct AAP_OVERRIDE_CMD (set AAP_PROJECT_GIT_BRANCH[, AAP_EE_IMAGE])
  Boot->>Refresh: run with COMPONENT_OVERRIDE_CMD + AAP_OVERRIDE_CMD
  Refresh->>Installer: apply sed overrides to kustomization.yaml

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related PRs

openshift/release#79304: Related modifications to the OSAC cluster-tool vmaas boot path and applied overrides during e2e-vmaas provisioning.

Suggested labels

rehearsals-ack

Suggested reviewers

danmanor
jhernand

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly and specifically summarizes the main change: adding an e2e-vmaas presubmit test for osac-aap, which aligns with the primary modifications to the CI configuration and boot script.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

omer-vishlitzky · 2026-05-16T17:19:38Z

/pj-rehearse pull-ci-osac-project-osac-aap-main-e2e-vmaas

openshift-merge-bot · 2026-05-16T17:19:40Z

@omer-vishlitzky: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

omer-vishlitzky · 2026-05-17T10:14:49Z

/pj-rehearse pull-ci-osac-project-osac-aap-main-e2e-vmaas

openshift-merge-bot · 2026-05-17T10:14:51Z

@omer-vishlitzky: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

omer-vishlitzky · 2026-05-17T16:53:34Z

/pj-rehearse pull-ci-osac-project-osac-aap-main-e2e-vmaas

openshift-merge-bot · 2026-05-17T16:53:36Z

@omer-vishlitzky: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

omer-vishlitzky · 2026-05-17T17:25:00Z

/pj-rehearse pull-ci-osac-project-osac-aap-main-e2e-vmaas

openshift-merge-bot · 2026-05-17T17:25:02Z

@omer-vishlitzky: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

omer-vishlitzky · 2026-05-17T23:48:22Z

/pj-rehearse pull-ci-osac-project-osac-aap-main-e2e-vmaas

openshift-merge-bot · 2026-05-17T23:48:25Z

@omer-vishlitzky: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

omer-vishlitzky · 2026-05-18T10:09:53Z

/pj-rehearse pull-ci-osac-project-osac-aap-main-e2e-vmaas

openshift-merge-bot · 2026-05-18T10:09:57Z

@omer-vishlitzky: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

omer-vishlitzky · 2026-05-18T15:15:07Z

/pj-rehearse pull-ci-osac-project-osac-aap-main-e2e-vmaas

openshift-merge-bot · 2026-05-18T15:20:10Z

@omer-vishlitzky: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

openshift-merge-bot · 2026-05-18T15:20:27Z

@omer-vishlitzky, pj-rehearse: unable prepare a candidate for rehearsal; rehearsals will not be run. This could be due to a branch that needs to be rebased. ERROR:

couldn't checkout base SHA 627d3d96b191de80f7bc4dd827c477afd1482ff3: error checking out "627d3d96b191de80f7bc4dd827c477afd1482ff3": exit status 128 fatal: unable to read tree (627d3d96b191de80f7bc4dd827c477afd1482ff3)

omer-vishlitzky · 2026-05-18T15:24:52Z

/pj-rehearse pull-ci-osac-project-osac-aap-main-e2e-vmaas

openshift-merge-bot · 2026-05-18T15:24:55Z

@omer-vishlitzky: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

omer-vishlitzky · 2026-05-18T16:51:02Z

/pj-rehearse pull-ci-osac-project-osac-aap-main-e2e-vmaas

openshift-merge-bot · 2026-05-18T16:51:05Z

@omer-vishlitzky: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

Add e2e-vmaas presubmit test for osac-aap PRs. Unlike other OSAC components, AAP has three independently-versioned artifacts: the execution environment image, playbooks synced from git, and config manifests. The standard single-image swap doesn't cover playbook sync. Build the EE image from the PR source, copy config/ into the installer, and embed the PR head SHA as a marker file. The boot script detects the marker and overrides AAP_PROJECT_GIT_BRANCH and AAP_EE_IMAGE in the kustomize overlay so AAP uses the PR's code for all three artifacts.

omer-vishlitzky · 2026-05-19T00:03:50Z

/pj-rehearse pull-ci-osac-project-osac-aap-main-e2e-vmaas

openshift-merge-bot · 2026-05-19T00:03:52Z

@omer-vishlitzky: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

openshift-merge-bot · 2026-05-19T00:06:54Z

[REHEARSALNOTIFIER]
@omer-vishlitzky: the pj-rehearse plugin accommodates running rehearsal tests for the changes in this PR. Expand 'Interacting with pj-rehearse' for usage details. The following rehearsable tests have been affected by this change:

Test name	Repo	Type	Reason
pull-ci-osac-project-osac-aap-main-e2e-vmaas	osac-project/osac-aap	presubmit	Presubmit changed
pull-ci-osac-project-osac-aap-main-images	osac-project/osac-aap	presubmit	Presubmit changed
pull-ci-osac-project-osac-test-infra-main-e2e-vmaas	osac-project/osac-test-infra	presubmit	Registry content changed
pull-ci-osac-project-osac-installer-main-e2e-vmaas	osac-project/osac-installer	presubmit	Registry content changed
pull-ci-osac-project-fulfillment-service-main-e2e-vmaas	osac-project/fulfillment-service	presubmit	Registry content changed
pull-ci-osac-project-osac-operator-main-e2e-vmaas	osac-project/osac-operator	presubmit	Registry content changed

Interacting with pj-rehearse

Comment: /pj-rehearse to run up to 5 rehearsals
Comment: /pj-rehearse skip to opt-out of rehearsals
Comment: /pj-rehearse {test-name}, with each test separated by a space, to run one or more specific rehearsals
Comment: /pj-rehearse more to run up to 10 rehearsals
Comment: /pj-rehearse max to run up to 25 rehearsals
Comment: /pj-rehearse auto-ack to run up to 5 rehearsals, and add the rehearsals-ack label on success
Comment: /pj-rehearse list to get an up-to-date list of affected jobs
Comment: /pj-rehearse abort to abort all active rehearsals
Comment: /pj-rehearse network-access-allowed to allow rehearsals of tests that have the restrict_network_access field set to false. This must be executed by an openshift org member who is not the PR author

Once you are satisfied with the results of the rehearsals, comment: /pj-rehearse ack to unblock merge. When the rehearsals-ack label is present on your PR, merge will no longer be blocked by rehearsals.
If you would like the rehearsals-ack label removed, comment: /pj-rehearse reject to re-block merging.

akshaynadkarni · 2026-05-19T01:03:48Z

/hold

akshaynadkarni

Left a couple of comments and added /hold. Comments are nits. PTAL prior to using /unhold.

openshift-ci · 2026-05-19T01:05:20Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: akshaynadkarni, omer-vishlitzky

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

~~ci-operator/config/osac-project/osac-aap/OWNERS~~ [akshaynadkarni]
~~ci-operator/jobs/osac-project/osac-aap/OWNERS~~ [akshaynadkarni]
~~ci-operator/step-registry/osac-project/cluster-tool/boot/OWNERS~~ [akshaynadkarni,omer-vishlitzky]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

omer-vishlitzky · 2026-05-19T07:55:58Z

/retest

omer-vishlitzky · 2026-05-19T08:03:34Z

/pj-rehearse pull-ci-osac-project-osac-aap-main-e2e-vmaas

openshift-merge-bot · 2026-05-19T08:03:37Z

@omer-vishlitzky: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

omer-vishlitzky · 2026-05-19T08:03:45Z

/test openshift-image-mirror-mappings

omer-vishlitzky · 2026-05-19T10:04:54Z

/pj-rehearse ack

openshift-merge-bot · 2026-05-19T10:04:57Z

@omer-vishlitzky: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

omer-vishlitzky · 2026-05-19T10:04:59Z

/unhold

openshift-ci · 2026-05-19T10:15:13Z

@omer-vishlitzky: all tests passed!

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

…ft#79365) Add e2e-vmaas presubmit test for osac-aap PRs. Unlike other OSAC components, AAP has three independently-versioned artifacts: the execution environment image, playbooks synced from git, and config manifests. The standard single-image swap doesn't cover playbook sync. Build the EE image from the PR source, copy config/ into the installer, and embed the PR head SHA as a marker file. The boot script detects the marker and overrides AAP_PROJECT_GIT_BRANCH and AAP_EE_IMAGE in the kustomize overlay so AAP uses the PR's code for all three artifacts.

openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label May 16, 2026

openshift-ci Bot requested review from adriengentil and danmanor May 16, 2026 12:09

omer-vishlitzky force-pushed the osac-853-aap-presubmit-e2e branch 2 times, most recently from 57679bb to 0557df8 Compare May 16, 2026 17:15

omer-vishlitzky force-pushed the osac-853-aap-presubmit-e2e branch 2 times, most recently from ed84f7c to 9a04463 Compare May 17, 2026 04:02

omer-vishlitzky force-pushed the osac-853-aap-presubmit-e2e branch 2 times, most recently from abf5e39 to 0ef5732 Compare May 18, 2026 15:11

omer-vishlitzky force-pushed the osac-853-aap-presubmit-e2e branch from 39a9c6f to 9697d15 Compare May 19, 2026 00:03

openshift-ci Bot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label May 19, 2026

akshaynadkarni approved these changes May 19, 2026

View reviewed changes

openshift-ci Bot assigned akshaynadkarni May 19, 2026

openshift-ci Bot added the lgtm Indicates that a PR is ready to be merged. label May 19, 2026

openshift-ci Bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label May 19, 2026

openshift-merge-bot Bot added the rehearsals-ack Signifies that rehearsal jobs have been acknowledged label May 19, 2026

openshift-ci Bot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label May 19, 2026

openshift-merge-bot Bot merged commit 5486e4b into openshift:main May 19, 2026
19 checks passed

coderabbitai Bot mentioned this pull request May 20, 2026

MGMT-22635: add CaaS CI workflow and e2e-caas jobs for OSAC #79512

Open

4 tasks

Conversation

omer-vishlitzky commented May 16, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Why AAP is different

Test plan

Uh oh!

openshift-ci-robot commented May 16, 2026 • edited by atlassian Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Why AAP is different

Test plan

Uh oh!

coderabbitai Bot commented May 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Reviews paused

Walkthrough

Changes

Sequence Diagram(s)

Estimated code review effort

Possibly related PRs

Suggested labels

Suggested reviewers

Uh oh!

omer-vishlitzky commented May 16, 2026

Uh oh!

openshift-merge-bot Bot commented May 16, 2026

Uh oh!

omer-vishlitzky commented May 17, 2026

Uh oh!

openshift-merge-bot Bot commented May 17, 2026

Uh oh!

omer-vishlitzky commented May 17, 2026

Uh oh!

openshift-merge-bot Bot commented May 17, 2026

Uh oh!

omer-vishlitzky commented May 17, 2026

Uh oh!

openshift-merge-bot Bot commented May 17, 2026

Uh oh!

omer-vishlitzky commented May 17, 2026

Uh oh!

openshift-merge-bot Bot commented May 17, 2026

Uh oh!

omer-vishlitzky commented May 18, 2026

Uh oh!

openshift-merge-bot Bot commented May 18, 2026

Uh oh!

omer-vishlitzky commented May 18, 2026

Uh oh!

openshift-merge-bot Bot commented May 18, 2026

Uh oh!

openshift-merge-bot Bot commented May 18, 2026

Uh oh!

omer-vishlitzky commented May 18, 2026

Uh oh!

openshift-merge-bot Bot commented May 18, 2026

Uh oh!

omer-vishlitzky commented May 18, 2026

Uh oh!

openshift-merge-bot Bot commented May 18, 2026

Uh oh!

omer-vishlitzky commented May 19, 2026

Uh oh!

openshift-merge-bot Bot commented May 19, 2026

Uh oh!

openshift-merge-bot Bot commented May 19, 2026

Uh oh!

akshaynadkarni commented May 19, 2026

Uh oh!

akshaynadkarni left a comment

Choose a reason for hiding this comment

Uh oh!

openshift-ci Bot commented May 19, 2026

Uh oh!

omer-vishlitzky commented May 19, 2026

Uh oh!

omer-vishlitzky commented May 19, 2026

Uh oh!

openshift-merge-bot Bot commented May 19, 2026

omer-vishlitzky commented May 16, 2026 •

edited by coderabbitai Bot

Loading

openshift-ci-robot commented May 16, 2026 •

edited by atlassian Bot

Loading

coderabbitai Bot commented May 16, 2026 •

edited

Loading